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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE (31 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)£3 Responsive to communication(s) filed on 18 February 2004 . 
2a)D This action is FINAL. 2b)£3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) KI Claim(s) t-27 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) £>3 Claim(s) 1,10 and 19 is/are rejected. 

7) [X] Claim(s) 2-9. 11-18. 20-27 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) [3 The drawing(s) filed on 18 February 2004 is/are: a)E3 accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to.by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) (3 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
aM AH b)D Some * c)D None of: 

1. £3 Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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Examiner's Detailed Office Action 

1. This Office is responsive to application 10/781471, filed February 18, 2004. 

2. Claims 1-27 have been examined. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed 
publication in this or a foreign country, before the invention thereof by the applicant for a patent. 

4. Claims 1 and 10 are rejected under 35 U.S.C. 102(a) as being anticipated by Lomker et al. 9 "A 
Multimodal System for Object Learning", 2002. 

Regarding claim 1. Lomker et al teach a learning system {see Abstract) comprising: dialog 
means for obtaining the name of an object from the user through a dialog with said user {see 
Abstract); plural recognizing means for detecting a plurality of feature data of said object 
respectively, and for recognizing the above object based on the above detection result and the 
learning result of said corresponding feature data of a known object previously stored {see 
Abstract, "The learning and retrieval process takes into account information gained from 
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multiple attributes calculated from an image recorded by a standard video camera...", Examiner 
interprets "multiple attributes" to be a plurality of feature data and "retrieval" to be of a known 
object previously stored, based on corresponding feature data.); storing means for storing 
relation information in that said name of said known object is connected with the recognition 
result of the above known object by each of said recognizing means (see §2 System overview, 
"If the judgement of the best judged hypothesis is above a threshold it is accepted as a found 
object and announced to the user by the speech synthesis module. Otherwise the object is learned 
in a dialog with the user. During this a new view with attribute features describing the object is 
generated and stored in the database. The database holds an unrestricted number of objects with 
an unrestricted number of views for every object and thus provides mappings between percepts 
and all symbols learned so far.", Examiner interprets the object name to be one of "all symbols 
learned" and the database to be the storing means.); and control means, if determining that said 
object is a new object based on the name of said object obtained by said dialog means, the 
recognition result of the above object by each of said recognizing means, and said relation 
information stored in said storing means, for making said needed recognizing means perform the 
learning of said corresponding feature data of the above object, and making said storing means 
newly store thus obtained relation information on the above object (see Fig. 1 and §2 System 
overview, Examiner interprets the "control module " to be the control means and that all storage 
and retrieval operations described in §2 are mediated by the control module.). 

Regarding claim 10. Lomker et al. teach a learning method (see Abstract) comprising: the first 
step for obtaining the name of an object from the user through a dialog with said user, and for 
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recognizing the above object based on the detection result of a plurality of feature data of said 
object and the learning results of said respective features of said known object previously stored 
(see §2 System overview, "If the control module receives a semantically parsed utterance with an 
object specification it grabs a new image and tries to find a corresponding object...", Examiner 
interprets "a semantically parsed utterance " to be the object name.)\ and the second step, if it is 
determined that said object is a new object based on the obtained name of said object, the 
recognition results based on said respective feature data of the above object, and relation 
information in that said name of said known object is connected with the recognition results of 
said respective feature data of the above known object, for performing the learning of said 
needed feature data of the above object, and newly storing thus obtained relation information on 
the above object (see §2 System overview, "If the judgement of the best judged hypothesis is 
above a threshold it is accepted as a found object and announced to the user by the speech 
synthesis module. Otherwise the object is learned in a dialog with the user. During this a new 
view with attribute features describing the object is generated and stored in the database.", 
Examiner interprets "the judgement of the best judged hypothesis " being below said threshold to 
mean said object is a new object. Examiner interprets "The database holds an unrestricted 
number of objects ... and ... provides mappings between percepts and all symbols learned so 
far. " to mean relation information in that said name of said known object is connected with the 
recognition results of said respective feature data of the above known object. Examiner 
interprets "Otherwise the object is learned in a dialog with the user. During this a new view with 
attribute features describing the object is generated and stored in the database. " to mean 
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performing the learning of said needed feature data of the above object, and newly storing thus 
obtained relation information on the above object.). 



Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth 
in section 102 of this title, if the differences between the subject matter sought to be patented and the prior 
art are such that the subject matter as a whole would have been obvious at the time the invention was made 
to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be 
negatived by the manner in which the invention was made. 

6. Claim 19 is rejected under 35 U.S.C. 103(a) as being unpatentable over Yoshizaki et ai 9 
"Mutual Assistance between Speech and Vision for Human-Robot Interface", 2002 in view of 
Lomker et al. . 

Regarding claim 19. Yoshizaki et al. teach a robot apparatus (see Abstract) comprising: dialog 
means for obtaining the name of an object from the user through a dialog with said user (see p. 
1312, §5. Assistance by Speech, "For example. Consider the case that the user asks "Bring that 
apple,". . .", Examiner interprets "The speech recognition result "apple "... " to be part of a 
dialog means for obtaining the name of an object.); plural recognizing means for detecting the 
predetermined different features of said object respectively, and for recognizing the above object 
based on the above detection result and the learning result of said corresponding feature of said 
known object previously stored (see p. 1312, §5. Assistance by Speech, "For example. Consider 
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the case that the user asks "Bring that apple. 5 '. Examiner interprets color and shape to be 
plural recognizing means of a known object previously stored), 

Yoshizaki et al do not teach a storing means for storing relation information in that said name of 
said known object is connected with the recognition result of the above known object by each of 
said recognizing means; and control means, if determining that said object is a new object based 
on the name of said object obtained by said dialog means, the recognition result of the above 
object by each of said recognizing means, and said relation information stored in said storing 
means, for making said needed recognizing means perform the learning of said corresponding 
feature of the above object, and making said storing means newly store thus obtained relation 
information on the above object. 

However, Lomker et al. do teach a storing means for storing relation information in that said 
name of said known object is connected with the recognition result of the above known object by 
each of said recognizing means (see 4., above); and control means, if determining that said object 
is a new object based on the name of said object obtained by said dialog means, the recognition 
result of the above object by each of said recognizing means, and said relation information stored 
in said storing means, for making said needed recognizing means perform the learning of said 
corresponding feature of the above object, and making said storing means newly store thus 
obtained relation information on the above object (see 4., above). 

It would have been obvious at the time the invention was made to persons having ordinary skill 
in the art to combine Yoshizaki et al with Lomker et al to add deictic gestures to speech to refer 
to objects. 
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Allowable Subject Matter 



7. Claims 2-9, 11-18, and 20-27 are objected to as being dependent upon a rejected base claim, 
but would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any intervening claims. 



Correspondence Information 

Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Nathan H. Brown, Jr. whose telephone number is 571-272- 8632. The 
examiner can normally be reached on M-F 0830-1700. If attempts to reach the examiner by 
telephone are unsuccessful, the examiner's supervisor, Anthony Knight can be reached on 571- 
272-3687. The fax phone number for the organization where this application or proceeding is 
assigned is 703-872-9306. Information regarding the status of an application may be obtained 
from the Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For more 
information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions 
on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217- 
9197 (toll-free). 
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Supennsory Patent Examiner 
Tech Center 2100 



Nathan H. Brown, Jr. 
May 16, 2006 



